SELA end2end #1621
SELA end2end #1621
Conversation
|
…y`, and modified the related calling functions, and updated the README.md.
metagpt/ext/sela/runner/sela.py
Outdated
| Extract and parse JSON content from the given string. | ||
| """ | ||
| try: | ||
| return json.loads(json_string[json_string.find("{") : json_string.rfind("}") + 1]) |
There was a problem hiding this comment.
you can ask the LLM to generate a ```json block and use metagpt.utils.common.CodeParser.parse_code to parse it
|
Great work and cannot wait to try it when ready. Just a general comment: can you make sure o1 is also fully supported with SELA? Because currently it seems o1 with metagpt's examples/customized_agents is having some combability issue. |
Thanks for your interest! We will test SELA with o1 and see if it works. Btw, SELA is able to achieve competitive performance even with deepseek, I assume you want to see if o1 can push up the limit further, or are there other considerations? |
|
@garylin2099 Thanks! Yeah cannot wait to test its limits! |
metagpt/ext/sela/runner/sela.py
Outdated
| "use_fixed_insights": False, | ||
| "low_is_better": False, | ||
| "start_task_id": 2, | ||
| "from_scratch": False, |
There was a problem hiding this comment.
for the default config, maybe set from_scratch to be True since it's an end-to-end task and no insights are pre-generated
metagpt/ext/sela/run_sela.py
Outdated
| "Report rmse on the eval data. Do not plot or make any visualizations." | ||
| ) | ||
|
|
||
| data_dir = "/home/coder/project/chenxin/MetaGPT/metagpt/ext/sela/data/SELA-datasets/05_house-prices-advanced-regression-techniques" |
| if datasets_dir.rpartition("/")[-1] == dataset["dataset"]: | ||
| data_path = datasets_dir | ||
| else: | ||
| data_path = os.path.join(datasets_dir, dataset["dataset"]) | ||
| split_datasets = { | ||
| "train": os.path.join(data_path, "split_train.csv"), |
There was a problem hiding this comment.
maybe use Path from pathlib instead of os.path
…y`, and modified the related calling functions, and updated the README.md.
# Conflicts: # metagpt/ext/sela/README.md # metagpt/ext/sela/data/dataset.py # metagpt/ext/sela/run_sela.py # metagpt/ext/sela/runner/sela.py
| Extract and parse JSON content from the given string using CodeParser. | ||
| """ | ||
| try: | ||
| json_code = CodeParser.parse_code("", json_string, "json") |
There was a problem hiding this comment.
you dont need try except, let the error be raise up
| from metagpt.llm import LLM | ||
| from metagpt.utils.common import CodeParser | ||
|
|
||
| SELA_INSTRUCTION = """ |
Features
Influence
Other